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One  of  the  objectives  of  this  project  is  to  develop  a  plan  for  extending  per- 
formance-structure  oriented  (PSO)  CAI  that  was  successfully  applied  to  teaching 
basic  skills  to  Radar  Intercept  Observers,  to  a  set  of  similar  jobs  in  Naval 
operations.  (U) 

In  this  technical  report,  the  approach  that  will  be  taker  to  the  development 

f  11  1S,P.lan  }s  octlined-  This  will  involve  expansion  of  the  steps  that  were 
followed  ^  the  production  of  the  RIO  trainer,  to  include  the  additional 
dimension  of  different  jobs  in  the  set  selected  for  consideration  Thus 
comparative  analyses  across  training  requirements  for  these  jobs  will  be’ 
ma  e,  to  identify  commonalities,  with  respect  to  knowledge  and  skill  require¬ 
ments,  instructional  operations  in  the  Instruct  and  Practice  modes,  task 
mixtures  in  the  top-level  performance  sequence,  and  response  recording  and 
analysis,  and  adaptive  control  requirements.  (U) 

The  theory  of  how  human  performance  is  organized  is  briefly  discussed,  since 
this  determines  the  organization  and  instructional  strategy  used  in  PSO  CAI 
these  features  of  PSO  CAI  are  then  described.  (U) 
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I.  INTRODUCTION 


In  earlier  work  (Rigney,  et  al.,  1973),  an  individual  skills  trainer 
for  Radar  Intercept  Observers,  using  performance-oriented  CAI,  was  developed 
and  was  evaluated  in  the  RIO  "basics"  school.  This  approach  to  teaching 
basic  skills  is  potentially  applicable  in  a  variety  of  contexts  in  Naval 
training.  This  extension  would  be  facilitated  by  a  systematic  examination 
of  the  systems  design  kinds  of  issues  in  matching  instructional  resources, 
particularly  instructional  operators,  to  the  structure  of  tasks  to  be 
learned  and  to  the  characteristics  of  student  populations.  This  report  will 
describe  the  approach  that  is  being  formulated  for  the  systematic  considera¬ 
tion  of  these  issues.  Since  there  have  been  many  general  feasibility 
studies  of  CAI,  a  number  of  detailed  RFP's  that  attempt  to  specify  require¬ 
ments  for  instructional  systems,  and  some  prior  research  on  the  design  of 
these  systems  (e.g.,  Carpenter,  et  al.,  1972),  it  is  necessary  to  describe 
the  specific  concerns  of  this  current  approach.  First,  it  deals  with 
performance-structure  oriented  CAI,  in  distinction  to  what  Carbonnel  (1971) 
called  ad-hoc  frame-oriented  CAI,  or  computer-administered  programmed 
instruction.  Second,  it  is  confined  to  a  relatively  small  group  of  Navy 
jobs  in  aviution  operations  which  hopefully  will  be  a  manageable  semple 
for  the  analytical  resources  available.  Third,  it  is  primarily  concerned 
with  the  matching  of  instructional  operators  with  types  of  content  found 
in  this  sample  of  jobs. 

Instructional  operators  are  ways  of  presenting  stimulus  material, 
and  operations  that  can  be  required  of  the  student,  that  will  facilitate 


learning  and  improve  retention.  In  the  present  concc..t‘,  they  consist  of 
operations  that  can  be  implemented  with  a  graphics  display,  light  pen, 
suitable  auxiliary  displays  and  response  entry  devices,  and  generative 
logic  for  creating  and  managing  the  interaction  with  the  student.  An 
example  of  an  instructional  operator  is  the  simultaneous  display  of  true 
and  relative  motion  between  bogey  and  fighter  during  an  intercept  problem, 
to  allow  the  student  to  relate  the  motion  in  the  relative  display  to  the 
motion  in  the  true  display.  Another  example  of  an  instructional  operator  is 
logic  that  automatically  repeats  a  problem  in  a  "free  trial"  mode  if  a 
student  fails  to  achieve  a  minimum  level  on  a  practice  problem. 

The  application  of  instructional  operators  in  relation  to  what  is  to  be 
taught  and  who  is  to  learn  it  is  the  key  to  increasing  the  effectiveness  of 
CAI.  According  to  this  conception,  instructional  operators  would  be  used 
during  the  instruct  and  practice  stages  in  the  instruct-practice-test 
sequence  that  is  basic  to  all  instruction,  and  would  be  managed  by  control 
operators  embedded  in  some  method  for  optimizing  individualized  instruction. 

Ideally,  task  and  content  analysis,  and  allocation  of  instructional 
operators  to  resulting  categories,  would  be  done  automatically  by  a  com¬ 
puter  program.  Clearly,  the  present  state  of  instructioral  technology  does 
not  provide  this  luxury.  To  do  this,  there  is  a  need  for  a  way  of  repre¬ 
senting  the  structures  and  operations  in  some  common  formalism  that  would 
be  suitable  for  translation  into  program  logic.  Graph  theory  immediately 
comes  to  mind  as  a  possibility.  There  are  in  the  literature  examples  of 
its  use,  in  one  or  another  forms,  to  represent  and  to  analyze  a  variety  of 
structures,  e.g.;  group  communications  patterns,  electronic  circuits,  and 
the  syntactic  aspects  of  language. 
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One  of  the  objectives  of  this  research  is  to  investigate  the  oossibi- 
lities  for  using  graph  theory  to  represent  and  to  manipulate  the  kinds  of 
structures  encountered  in  planning  CAI  applications.  This  investigation 
will  be  pushed  only  as  far  as  necessary  to  identify  the  requirements  and 
to  make  a  decision  regarding  the  potential  payoffs.  If  it  appears  that  it 
would  be  fruitful  to  use  graph  theory  to  assist  in  developing  the  applica¬ 
tion  model  for  the  sample  of  Nc  al  jobs,  this  will  be  done.  Otherwise, 
more  conventional  methods  will  be  used. 

Other  important  issues  for  planning  the  application  of  CAI  to  a  spectrum 
of  jobs  are  the  optimization  of  individualized  instruction,  which  includes 
instructional  sequencing  and  control;  and  cost-effectiveness.  Atkinson 
and  Paulson  (1972)  have  discussed  the  requirements  for  optimizing  indivi¬ 
dualized  instruction,  have  described  several  methods  they  have  used,  and 
have  made  it  clear  that  estimating  cost-effectiveness  depends  on  the  con¬ 
sumer's  value  judgments  of  the  worth  of  alternative  objectives.  These 
other  issues  are  considerations  for  any  CAI  application,  and  since  they 
are  treated  so  ably  by  Atkinson  and  others,  they  will  not  be  a  major  em¬ 
phasis  of  this  work.  The  individual  trainer  developed  for  the  RIO  already 
uses  simple  adaptive  logic.  This  will  be  refined  as  part  of  the  further 
development  of  this  trainer.  Some  crude  cost  comparisons  can  be  made  with 
the  other  existing  device  for  RIO  basic  procedural  training.  Keep  in  mind 
that  this  existing  device  is  oriented  more  toward  radar  operator  training 
and  that  the  BTL  trainer  is  oriented  more  toward  procedural  training  and 
thus  the  sets  of  procedures  each  teaches  overlap  but  are  not  identical. 

This  device  costs  approximately  10  times  as  much,  requires  an  instructor 
to  operate  it,  does  not  record  and  analyze  student  responses,  is  very 
./pensive  to  modify,  breaks  down  frequently,  and  provides  less  "on-line" 
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practice  time  to  each  student.  Since  it  does  not  record  and  analyte  student 
responses,  there  is  no  basis  for  optimising  learning,  nor  any  basis  for 
estimating  its  effectiveness. 


II.  PERFORMANCE -ORIENTED  CAI 

Properly  controlled  practice  results  in  increased  proficiency.  This 
may  be  the  fundamental  law  of  training.  Its  power  is  demonstrated  every¬ 
day  in  literally  thousands  of  different  settings.  Great  fluency  in  per¬ 
formance  is  produced  only  by  large  amounts  of  properly  controlled  practice. 
This  may  be  the  second  law  of  training.  Of  course,  the  key  phrase  in 
these  statements  is  "properly  controlled."  Repetition  provides  the  context 
in  which  mediating  processes  can  occur.  These  must  be  directed  by  the 
manipulation  of  conditions  external  to  the  learner,  e.g.,  by  instructional 
operators.  Improperly  controlled  practice  at  the  least  may  be  inefficient. 
At  the  worst  it  may  be  disastrous.  This  also  is  demonstrated  every  day. 

Despite  the  universality  of  these  laws,  it  is  not  always  easy  or  even 
feasible  in  some  training  environments  to  provide  opportunities  for  properly 
controlled  practice  to  each  student  in  amounts  sufficient  to  bring  him  up 
to  a  desirable  level  of  fluency.  PSO  CAI  was  developed  to  provide 
individualized,  properly-controlled  practice  as  automatically  as  possible. 
That  is,  it  should  bring  the  Individual  student  up  to  some  desired  level 
of  proficiency  without  requiring  the  constant  attention  of  an  instructor 
during  the  practice  period,  and  without  requiring  an  operator  in  addition 
to  the  student  during  the  practice  period. 

In  this  section,  we  will  discuss  the  assumptions  about  the  organiza¬ 
tion  of  human  performance  that  are  the  "theoretical"  basis  for  PSO  CAI, 
and  describe  procedures  used  in  its  development,  to  provide  a  background 
for  the  following  sections. 
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The  Organization  of  Performance 


The  view  of  performance  as  a  highly  organized  serial  mixture  of  subgoals 
and  associated  action  clusters,  sustained  by  groups  of  mediating  processes, 
has  been  described  elsewhere  (e.g.,  DeCreene,  1970).  This  is  essentially 
an  information-processing  conception  derived  from  the  literature  of  informa¬ 
tion-processing  (Reitman,  1965)  cognitive  (Neisser,  1967),  and  cybernetic 
(Pask,  1970)  psychology  in  general,  and  from  recent  literature  on  mediating 
processes  in  learning  and  memory  (e.g.,  Tulving  and  Donaldson,  1972; 

Carroll  and  Freedle,  1972;  Sheehan,  1972;  Glaser,  1971).  Some  of  the  major 
kinds  of  mediating  processes  that  are  required  to  sustain  performance  at 
its  highest  level  are  at  least  suggested  in  the  following  figure: 


The  organization  of  serial-action  performance 


We  suggest  that  many  mediating  processes  in  each  box  in  the  figure 
are  learned,  and  that  this  learning  consists  essentially  of  elaboration  of 
complex  processes  from  some  sort  of  primitive  substrate  of  biologically 
determined  processes,  perhaps  common  to  all  mammalian  forms  of  life. 
According  to  this  view  of  the  organization  of  performance,  it  can  occur 
at  least  at  two,  and  probably  at  several  levels  of  serial  integration.  At 
the  highest  level,  the  performer  generates  the  series  of  activities  involved 
by  a  self-program  that  is  flexible  enough  to  meet  the  contingencies  of  the 
immediate  situation.  Perhaps  this  self-program  is  a  skeleton  routine  that 
is  filled  out  with  one  of  several  available  subroutines  at  each  point  in 
the  performance,  somewhat  in  advance  of  the  activities  generated  by  that 
subroutine.  The  requirement  for  this  extemporaneous  self -programming 
would  depend  upon  the  unpredictable  variability  in  the  performance  require¬ 
ments.  When  he  can  stay  at  this  highest  level,  the  performer  is  essentially 
through  with  training.  The  fact  that  he  can  perform  autonomously  and  meet 
criteria  for  proficiency  indicates  that. 

Before  this  time,  the  performer  is  dependent  on  external  instruction. 

He  may  be  able  to  sustain  his  performance  at  the  top  level  only  some  of  the 
time.  In  between,  he  must  concern  himself  with  learning  some  needed  sub¬ 
skill  he  did  not  know,  or  with  learning  some  needed  information  he  did  not 
have,  or  with  learning  control  processes  necessary  to  string  subskills 
together  into  the  correct  serial  mixture,  to  monitor  his  performance  for 
errors  and  for  correct  timing,  etc. 

There  is  little  doubt  that  this  kind  of  learning  is  partially  dependent 
upon  self-organizing  processes.  We  do  not  know,  except  by  indirect  and 
usually  ambiguous  observations,  much  about  the  exact  nature  of  mediating 
processes.  Those  chat  we  do  know  something  about  as  a  consequence  of 
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laboratory  research  possibly  are  only  "surface"  manifestations  of  even  more 
obscure  processes  which  tend  to  be  self-organizing.  On  the  other  hand,  as 
both  Bruner  (1966)  and  Pask  (1970)  have  pointed  out,  in  the  early  stages  of 
learning  the  student  is  not  in  a  position  to  instruct  himself.  He  has 
entered  into  a  kind  of  contract  with  the  instructor,  human  or  machine,  which 
includes  the  idea  that  the  instructor  will  be  responsible  for  providing 
effective,  high  quality  instruction,  until  the  studer*-  has  learned  enough 
to  become  more  self-directed. 

An  Integrated  Approach  to  Teaching 
Performance  with  CAT 

If  the  above  views  of  the  nature  of  performance  are  correct,  any 
method  for  teaching  students  how  to  perform  by  giving  them  properly  con¬ 
trolled  practice  must  be  sensitive  to  structures;  to  the  serial  patterns  of 
performance,  to  the  levels  of  integration,  to  task  structures,  and  to,  where 
these  are  involved,  device  structures. 

One  way  to  do  this  is  to  simulate  essential  characteristics  of  the 
performance  situation,  providing  a  series  of  graded  practice  problems, 
procedures  for  moving  up  and  down  across  levels  of  organization,  adaptive 
control  sensitive  to  individual  differences,  instructional  operators 
matched  to  the  different  learning  requirements,  and  sufficiently  detailed 
response  analynis  tc  guide  adaptive  control  and  to  provide  a  description 
of  how  well  the  system  is  working.  Of  course,  job  or  task  simulators  have 
been  around  for  a  long  time,  but  they  have,  in  most  cases,  been  woefully 
incomplete.  The  major  emphasis  in  their  design  usually  has  been  on 
fidelity  of  simulation,  while  adaptive  control,  instructional  operators, 
movement  across  levels  of  performance  organization,  and  detailed  response 
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recording  and  analysis  usually  have  been  altogether  absent.  Simulation 
alone  does  not  provide  adequate  control  over  practice. 

Some  advantages  of  this  approach  are: 

1.  The  performance  i£  the  criterion  test.  Students  are  "taking  the 
test"  while  practicing. 

2.  Students  can  iterate  back  and  forth  between  the  top  level  and  lower 
levels,  receiving  extra  drill  on  subskills  in  which  they  are  weak 
and  returning  to  the  top  level  to  try  again  to  perform  at  that 
level. 

3.  No  time  is  wasted  in  presenting  material  the  student  already 
knows.  If  he  can  do  it,  he  is  automatically  transitioned  by  the 
program  logic  to  more  difficult  material. 

A.  The  student  learns  self-monitoring  and  self-programming  processes 
needed  in  the  context  of  the  actual  performance,  as  well  as  the 
information-processing  skills  that  usually  are  the  sole  concern 
of  instruction. 

Steps  ir.  the  Development  of  Performance- 
Oriented  CAI 

The  procedures  followed  for  developing  this  type  of  CAI  for  one 
training  situation  can  be  simpler  than  if  it  is  to  be  extended  to  a  set  of 
situations.  Then  it  is  worthwhile  considering  questions  of  what  aspects  of 
the  development  can  be  generalized  and  how  much.  Under  certain  circumstances, 
generative  programming  can  be  used  to  reduce  the  labor  of  preparing  instruc¬ 
tional  sequences.  Commonalities  can  be  sought  among  mediating  skills, 
content,  and  instructional  operators. 


The  general  steps  that  were  followed  for  the  development  of  the  RIO 
program  are: 

1.  Achieve  a  thorough  understanding  of  the  serial  composition  of  the 
performance  to  be  taught. 

2.  Analyze  each  different  task  in  the  serial  string  to  a  depth  suffi¬ 
cient  to  reach  the  probable  lower  bound  of  the  student  population's 
entering  skills  and  knowledge. 

3.  Identify  elements  in  the  task  hierarchy  that  require  special 
instructional  operators  and  develop  specifications  for  these. 

4.  Produce  an  instructional  flowchart  for  the  interactions  between 
student  and  program.  This  is  the  instrument  for  communicating 
between  the  instructional  technologist  and  the  computer  programmer. 

5.  Produce  program  flowcharts  and  write  the  computer  program. 

6.  Concurrently,  produce  the  data  base  for  the  program. 

Steps  4  and  5  typically  will  require  several  iterations  before  the 

instructional  flowchart  can  be  translated  into  program  flowcharts,  since 

the  two  are  quite  different.  The  programmer  has  to  understand  what  is 

wanted  in  the  student-program  interaction  and  then  has  to  produce  what 

« 

amount  to  entirely  different  flowcharts. 

Step  2  deals  with  an  important  issue,  what  might  be  called  the 
"instructional  bandwidth"  of  a  CAI  system.  What  assumptions  are  made 
about  the  levels  of  entering  skills  possessed  by  the  student  population? 

How  wide  a  range  of  skills  must  the  system  be  designed  to  teach?  PSO  CAI 
moves  a  student  to  a  lower  level  of  performance,  to  practice  some  sub¬ 
skill,  for  some  relatively  short  time,  then  the  student  tries  to  perform 
at  the  higher  level  again.  Clearly,  in  this  drill  and  practice  mode, 
it  is  not  the  purpose  of  the  drill  levels  to  give  lengthy  courses  in 
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fundamental  skills.  There  must  be  some  cutoff  point.  Since  PSO  CAI  is 
designed  to  be  used  in  courses  in  which  other  training  methods  impart 
"theory,"  the  student  population  is  likely  to  have  been  selected  already. 
These  selection  criteria  usually  would  be  suitable  for  the  entering 
bandwidth  of  the  CAI  system. 

These  general  steps  will  be  followed  in  the  development  of  a  model  for 
extending  this  type  of  CAI  to  a  range  of  different  training  requirements. 

An  initial  classification  scheme  will  be  used  to  expand  each  step  by  adding 
the  dimension  of  different  jobs,  so  that  a  two-dimensional  array,  rather 
than  a  single  vector  of  elements  will  result  from  the  work. 
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